Systems and methods for automated healthcare services

ABSTRACT

Healthcare services can be automated utilizing a system that recognizes at least one characteristic of a patient based on images of the patient acquired by an image capturing device. Relying on information extracted from these images, the system may automate multiple aspects of a medical procedure such as patient identification and verification, positioning, diagnosis and/or treatment planning using artificial intelligence or machine learning techniques. By automating these operations, healthcare services can be provided remotely and/or with minimum physical contact between the patient and a medical professional.

BACKGROUND

Conventional healthcare services generally require close contact betweena patient and a medical professional. In radiation therapy and medicalimaging, for instance, a doctor or technician usually needs to bepresent in the treatment room during at least the positioning stage toensure that the patient get into a desirable position or pose to allowthe treatment or scan to proceed in a precise and accurate manner. Thesetraditional methods of providing healthcare services are manual innature and suffer from human errors, inconsistencies and lack ofreal-time monitoring capabilities. At the same time, close contactbetween patients and medical professionals may lead to increased risksof cross-contamination, unintended exposure to radiation, and/or violentincidents ranging from verbal abuse to physical assaults.

SUMMARY

Described herein are systems, methods and instrumentalities forproviding automated (e.g., remote and/or contactless) healthcareservices to a patient. In an example implementation, an automatedhealthcare system may include a sensing or image capturing deviceinstalled in a medical environment and configured to capture images of apatient. The images may be transmitted to or retrieved by a control unitand used to determine at least one characteristic of the patient. Inexamples, the physical characteristics and/or identity of the patientmay be determined by analyzing the images (e.g., at a pixel level) usingartificial intelligence methods and/or machine-learned models. Relyingon the identity and/or the at least one characteristic of the patientdetermined via such methods or models, the control unit mayautomatically complete one or more aspects of a medical procedure forthe patient, for example, remotely and/or without requiring close humancontact with the patient. In examples, the automated aspects may includeremotely controlling a medical device or remotely providing instructions(e.g., positioning instructions) to the patient in connection with themedical procedure based on the characteristic of the patient extractedfrom the images of the patient. In examples, the automated aspects mayinclude determining the readiness of the patient for the medicalprocedure and/or movements of the patient before and during the medicalprocedure based on the images collected by the image capturing device.In examples, the automated aspects may include determining a spatialrelationship between the image capturing device and a medical device(e.g., a medical scanner) and complete one or more aspects of themedical procedure based on the spatial relationship. For example, thespatial relationship may be used to determine a parameter (e.g., a scanparameter) associated with the medical procedure and the one or moreimages of the patient may be overlaid with an indication of theparameter to facilitate decision making. In examples, the automatedaspects may also include providing instructions to the patient orfeedback to a medical professional (e.g., located remotely from thepatient) regarding the status of the patient or the progression of themedical procedure.

The images of the patient described herein may include photos of thepatient taken by a camera or thermal images of the patient generated bya thermal sensor. The at least one characteristic of the patientdetermined from these images may include the height, weight, and/or bodyshape of the patient. In addition, the automation of the medicalprocedure may be further assisted by patient information retrieved fromother sources including, for example, a medical record repository. Suchinformation may be used to verify the identity of the patient and/or toautomatically determine the protocols or parameters associated with themedical procedure.

BRIEF DESCRIPTION OF THE DRAWINGS

A more detailed understanding of the examples disclosed herein may behad from the following description, given by way of example inconjunction with the accompanying drawing.

FIG. 1 is a simplified diagram illustrating an example system forproviding automated healthcare services as described herein.

FIG. 2 is a simplified block diagram illustrating an example controlunit as described herein.

FIG. 3 is a simplified block diagram illustrating functional modules ofan example control unit as described herein.

FIG. 4 is a flow diagram illustrating a method that may be implementedby the automated healthcare system depicted in FIG. 1.

DETAILED DESCRIPTION

The present disclosure is illustrated by way of example, and not by wayof limitation, in the figures of the accompanying drawings.

FIG. 1 is a diagram illustrating an example system 100 for providingautomated healthcare services at a medical facility such as a hospital.The healthcare services may include, for example, a medical scanprocedure conducted via a medical scanner 102 (e.g., a computertomography (CT) scanner, a magnetic resonance imaging (MRI) machine, apositron emission tomography (PET) scanner, an X-ray machine, etc.), ora radiation treatment procedure delivered through a medical linearaccelerator (LINAC) (not shown). One or more aspects of the healthcareservices may be automated through the system 100. For example, thesystem 100 may include at least one sensing or image capturing device104 configured to capture one or more images of a patient 106 in oraround a medical environment (e.g., a hospital, a physician's office, ascan or treatment room, etc.). The image capturing device 104 maycomprise one or more cameras digital color cameras, 3D cameras, etc.),one or more sensors (e.g., red, green and blue (RGB) sensors, depthsensors, thermal sensors, infrared (FIR) or near-infrared (NIR) sensors,radar sensors, etc.), and/or other types of sensing devices configuredto detect the patient's presence and generate one or more imagesdepicting the patient in response. Depending on the type of sensors orsensing devices used, the images generated by the imaging capture device104 may include, for example, a photo of the patient taken by a camera,a thermal image of the patient generated by a thermal sensor, a radarimage of the patient produced by a radar sensor, and/or the like.Further, although the image capturing device 104 is described herein asbeing configured to take a picture or image of the patient, the imagecapturing device 104 may also be a scanner configured to obtain imagesof the patient based on an existing photo or picture of the patient(e.g., a driver's license presented by the patient during check-in).

The image capturing device 104 may be configured to be installed invarious locations of the medical environment such as inside ascan/treatment room, around a registration desk, in a hallway, on themedical scanner 102, etc. From the installation location, the imagecapturing device 104 may capture an image of the patient from a certainviewpoint or angle. The viewpoint or angle of the image capturing device104 may be adjusted (e.g., by sending a control signal to the device orby manually adjusting the orientation of the device) so that multipleviews or images of the patient may be taken from different viewpoints orangles using a single image capturing device. Alternatively, multipleimage capturing devices may be included in the system 100 to captureimages of the patient from different angels or viewpoints.

In example implementations, a first image capturing device (e.g., afirst instance of the image capturing device 104) may be installed at alocation (e.g., at a registration desk or kiosk) to capture an image ofthe patient upon the patient's arrival at a medical facility. The imagemay then be used to identify the patient, determine a medical procedurescheduled for the patient based on the identity and direct the patientto an appropriate treatment or scan room 108 for receiving the medicalprocedure. In example implementations, a second image capturing device(e.g., a second instance of the image capturing device 104) may beinstalled in the treatment or scan room 108 to capture images of thepatient before and/or during a medical procedure. The images may then beused to determine at least one characteristic of the patient, todetermine or verify the identity of the patient, to determine and/oradjust (e.g., automatically and/or remotely) one or more operatingparameters associated with the medical procedure, and/or to monitor theactivities or status of the patient before and during the medicalprocedure.

The system 100 may further include a control unit 110 configured toreceive and process the images of the patient produced by the imagecapturing device 104. The control unit 110 may be located in the sameroom 108 as the image capturing device 104 and/or the patient 106. Thecontrol unit 110 may also be located remotely from the image capturingdevice 104 or the patient 106, for example, in a room 112 (e.g., acontrol room) that is separate or isolated from the room 108 (e.g., therooms 108, 112 may be located on different floors or in differentbuildings). Regardless of its location, the control unit 110 may becommunicatively coupled to the image capturing device 104, for example,over a communication network 114 (e.g., a wired or wirelesscommunication network). In examples, the control unit 110 may beconfigured to retrieve or receive images from the image capturing device104 over the communication network 114 on a periodic basis (e.g., onceevery of minute, according to a predetermined schedule, etc.), Inexamples, the control unit 110 may be configured to receive anotification from the image capturing device 104 when an image hasbecome available and to retrieve the image from the sensing device inresponse to receiving the notification.

Once received or retrieved by the control unit 110, the images of thepatient may be used to automate one or more aspects of a healthcareservice for the patient. For example, in response to receiving theimages from the image capturing device 104, the control unit 110 mayanalyze the images (e.g., at a pixel level such as pixel by pixel, ingroups of pixels, etc.) to extract a plurality of features thatcollectively indicate the identity of the patient. In examples, thecontrol unit 110 may determine the identity of the patient by comparingthese extracted features against a set of known features of the patientstored in a feature database. In examples, the control unit 110 mayutilize an artificial neural network trained for visual recognition toextract the features and determine the identity of the patient. Theneural network may be a convolutional neural network (CNN) thatcomprises a cascade of layers each trained to make pattern matchingdecisions based on a respective level of abstraction of the visualcharacteristics contained in the images (e.g., in the pixels of theimages). The training of the neural network may be performed using largeamounts of imagery data and/or specific loss functions through which theneural network may learn to extract features in the form of featurevectors) from a newly provided input image, determine whether thefeatures match those of a known person, and indicate the matchingresults at an output of the neural network. Example implementations ofvisual recognition and neural networks will be described in greaterdetail below.

In addition to the identity of the patient, the control unit 110 mayalso be configured to determine one or more characteristics (e.g.,physical characteristics) of the patient based on the images produced bythe image capturing device 104. These characteristics may include, forexample, height, weight, body shape, pose, age, and/or gender of thepatient that may be used to facilitate the automation of healthcareservices. For example, the characteristics may be used to verify theidentity of the patient against other sources of information regardingthe patient. These sources of information may include, for instance, amedical record repository 116 (e.g., one or more medical recorddatabases) configured to store patient medical information such as thepatient's general information (e.g., patient ID, name, address, weight,height, age, gender, etc.), diagnostic and treatment history, imagerydata associated with a past medical procedure, etc. The repository 116may be communicatively coupled to the control unit 110 via thecommunication network 114 or a different communication network. As such,the control unit 110 may, in response to determining the identity and/orthe characteristics of the patient based on the images acquired from theimage capturing device 104, retrieve all or a subset of the patientinformation from the repository 116 and cross-check the retrievedpatient information (e.g. height, weight, gender, age) against thecharacteristics of the patient determined from the images to identifypotential errors in the identification.

The characteristics of the patient described herein may also be used toconfigure or adjust a medical procedure for the patient. For example,upon determining the identity of the patient, the control unit 110 maybe further configured to determine that a certain medical procedure(e.g., a chest X-ray, a CT scan, etc.) is to be performed for thepatient, and retrieve information about the medical procedure byquerying scheduling and/or medical history information stored in therepository 116. The retrieved information may include operatingparameters associated with the medical procedure such as scan locations,scan directions, and/or scan ranges, which may be comprised in aprotocol designed for the medical procedure. Based on the parameterinformation and the characteristics of the patient, the control unit 110may proceed to configure the medical equipment (e.g., the scanner 102)involved in the medical procedure. For example, the control unit 100 maydetermine, based on the scan location information indicated in a scanprotocol and the height of the patient, that one or more adjustments(e.g., adjustments to the height of a scan bed or a scan direction) needto be made to the scanner 102, and subsequently transmit a controlsignal to the scanner to effectuate the adjustments. The control signalmay include a di and/or analog control signal, and may be transmitted tothe scanner via wired or wireless means.

The control unit 110 may determine the parameters associated with amedical procedure or a medical device based on a spatial relationshipbetween the at least image capturing device and the medical device(e.g., a medical scanner). For example, the at least one image capturingdevice may be associated with or characterized by a first coordinatesystem, and the images produced by the at least one image capturingdevice may define objects captured in the images using the firstcoordinate system. The medical device, on the other hand, may beassociated with or characterized by a second coordinate system that isdifferent from the first coordinate system (e.g., in terms of originsand/or orientations). The control unit 110 may be configured todetermine the spatial relationship between the first and secondcoordinate systems, and, when necessary, convert the coordinates of anobject (e.g., contained in an image of the patient) in the firstcoordinate system to corresponding coordinates in the second coordinatesystem. As such, a location of the object relative to the medical device(e.g., in the second coordinate system) may be determined based on thelocation of the object indicated by the image capturing device (e.g., asdefined in the first coordinate system). The location information maythen be used to determine or adjust parameters associated with themedical procedure or the medical device (e.g., a position and/ororientation of a medical scanner). In examples, the control unit may beconfigured to overlay the one or more images of the patient with anindication of the parameters determined based on the spatialrelationship described herein, and cause a representation of theoverlaid images to be displayed to a medical professional (e.g., tofacilitate medical decision making based on the images captured by theimage capturing device).

The control unit 110 may use the images provided by the image capturingdevice 104 to monitor the status of the patient before and during amedical procedure. For example, the control unit 110 may be configuredto evaluate the readiness of the patient by collecting multiple imagesof the patient over a certain period of time, extracting positionalinformation of the patient from each of the collected images, andcomparing the positional information in the multiple images to ensurethat the patient has remained steady in a desirable position or pose forthe medical procedure. In another example, the control unit 110 may beconfigured to recognize activities of the patient by analyzing multipleimages collected by the image capturing device 104 to determine whetherthe patient has followed instructions (e.g., positioning instructions)provided to the patient.

Information and/or instructions generated by the control unit 110 may bepresented to the patient 106 in various forms. In an exampleimplementation, the system 100 may include a display device 118 locatedin the treatment room 108. The display device 118 may include one ormore monitors (e.g., computer monitors, TV monitors, tablets, mobiledevices such as smart phones, etc.), one or more speakers, one or moreaugmented reality (AR) devices (e.g., AR goggles), and/or otheraccessories configured to facilitate audio or visual representation. Thedisplay device 118 may be communicatively coupled to the control unit110 via the communication network 114 or another suitable communicationlink. As described herein, the information or instructions presented viathe display device 118 may include desired positions and poses for anupcoming medical procedure, positions taken by the patient during pastscans, adjustment instructions for the patient to get into the desiredpositions or poses, etc. The information and/or instructions may bepresented to the patient 106 in various formats including, for example,videos, animations, AR presentations, etc.

Information and/or instructions generated by the control unit 110 mayalso be presented (e.g., remotely) to a medical professional overseeingthe patient or the medical procedure. The medical professional may belocated remotely from (e.g., isolated from) the patient 106, e.g., inthe room 112. The information presented to the medical professional mayinclude the images captured by the at least one image capturing device,feedback information regarding the current position or pose of thepatient 106, a medical history of the patient 106, the current operatingparameters or state of the medical scanner 102, etc. In examples, thefeedback may include information (e.g., patient position information)synthesized by the control unit 110 based on images captured frommultiple viewpoints or angles (e.g., by multiple instances of the imagecapturing device 104). The medical professional may use the multi-viewinformation to visually inspect to the patient and/or the medicalequipment in room 108 to ensure that the patient and equipment areindeed ready for an upcoming procedure. The information described hereinmay be presented via a display device attached to the control unit 110or via a separate display device (not shown in FIG. 1) isolated from thepatient (e.g., in a separate room from where the patient is). Thedisplay device may include one or more monitors (e.g., computermonitors, TV monitors, tablets, mobile devices such as smart phones,etc.), one or more speakers, one or more augmented reality (AR) devices(e.g., AR goggles), and/or other accessories configured to facilitateaudio or visual representation. The display device may becommunicatively coupled to the control unit 110 via the communicationnetwork 114 or another suitable communication link. The informationand/or instructions may be presented to the medical professional invarious formats including, for example, videos, animations, ARpresentations, etc.

The system 100 may also include devices for a medical professional toprovide inputs or instructions to the system 100 or the patient 106.Such inputs or instructions may relate to adjusting the position of thepatient or the operating parameters of scanner 102, confirming thereadiness of the patient or the scanner, initiating a medical procedureafter the readiness is confirmed, etc. Suitable input devices foraccomplishing these tasks may include a keyboard, a mouse, avoice-controlled input device, a touch sensitive input device (e.g., atouch screen), and/or the like. The input devices may be attached to thecontrol unit 110 or may be separate from the control unit 110.

In example implementations, the control unit 110 may also be configuredto make automatic diagnosis for the patient 106 based on informationcollected during the medical procedure (e.g., based on scan images ofthe patient collected during the procedure), physical characteristics ofthe patient and/or a medical history of the patient. For example, thecontrol unit 110 may utilize artificial intelligence (e.g., a neuralnetwork trained for medical image recognition) to identify abnormalitiesin the scan images of the patient and the medical conditions that may beindicated by the abnormalities. The control unit 110 may furtherprioritize and/or label each scan image or diagnostic finding ascritical, urgent, non-urgent, normal or uncertain, and generate a reportfor the diagnoses and/or prioritization. The control unit 110 mayprovide the report to a medical professional for further analysis orinvestigation. The control unit 110 may also provide additionalinstructions to the patient 106 based on the diagnoses. For example, thecontrol unit 110 may instruct the patient to leave the treatment room108 if the diagnosis is negative or to schedule additional procedures ifthe diagnosis is ambiguous or positive. Example implementations ofAI-based medical diagnosis will be described in greater detail below.

Using the system 100, a healthcare service provider may be able tomonitor and control multiple treatment or scan rooms (e.g., the room108) simultaneously, e.g., from one control room (e.g., the room 112).The control room may host a control unit (e.g., the control unit 110)communicatively coupled to multiple image capturing devices (e.g., theimage capturing device 104) and configured to receive images provided bythe image capturing devices. Based on the images, the control unit maybe able to carry out multiple operations associated with automating ahealthcare service (e.g., patient identification/verification, patientreadiness detection, patient positioning, etc.). One or more of theseoperations may be performed by the control unit in parallel, andfeedback may be provided to a medical professional to ensure theoperations proceed in a desired manner (e.g., the medical professionalmay intervene in the automated process if necessary).

It should be noted that one or more of the operations or tasks describedherein as being executed by the control unit 110 may also be performedby another device or component of the system 100 such as the imagecapturing device 104. For example, the image capturing device 104 may beconfigured with the necessary computing power or logic for determiningthe identity or characteristics of the patient, or adjusting theoperating parameters (e.g., the height of a scan bed) of a medicaldevice.

FIG. 2 is a simplified block diagram illustrating an example controlunit 200 (e.g., the control unit 110 in FIG. 1) as described herein. Thecontrol unit 200 may operate as a standalone device or may comprisemultiple interconnected (e.g., networked or clustered) devicesconfigured to jointly perform the functions described herein. Inexamples of a networked deployment, the control unit 200 may operate inthe capacity of a server device or a client device, or it may act as apeer device in peer-to-peer (or distributed) network environments.Further, while only a single unit is shown in FIG. 2, the term “controlunit” shall be taken to potentially include multiple units or machinesthat individually or jointly execute a set of instructions to performany one or more of the functions discussed herein. The multiple units ormachines may be hosted in one location or multiple locations, forexample, under a distributed computing architecture.

The control unit 200 may include at least one processor 202 which inturn may include one or more of a central processing unit (CPU), agraphics processing unit (GPU), a microcontroller, a reduced instructionset computer (RISC) processor, application specific integrated circuits(ASICs), an application-specific instruction-set processor (ASIP), aphysics processing unit (PPU), a digital signal processor (DSP), a fieldprogrammable gate array (FPGA), or any other circuit or processorcapable of executing the functions described herein. The control unit200 may further include a communication circuit 204, a memory 206, amass storage device 208 and/or an input device 210 interconnected witheach other and the processor 202 via a communication link 214 (e.g., anaddress and/or data bus). The communication circuit 204 may beconfigured to transmit and receive information utilizing one or morecommunication protocols (e.g., TCP/IP) and one or more communicationnetworks including a local area network (LAN), a wide area network(WAN), the Internet, a wireless data network (e.g., a Wi-Fi, 3G, 4G/LTE,or 5G network). The memory 206 may include a machine-readable mediumconfigured to store instructions that, when executed, cause theprocessor 202 to perform one or more of the functions described herein.Examples of a machine-readable medium may include volatile ornon-volatile memory including but not limited to semiconductor memory(e.g., electrically programmable read-only memory (EPROM), electricallyerasable programmable read-only memory (EEPROM)), flash memory, and/orthe like. The mass storage device 208 may include one or more magneticdisks such as internal hard disks, removable disks, magneto-opticaldisks, CD-ROM or DVD-ROM disks, etc., on which instructions and/or datamay be stored to facilitate the performance of the functions describedherein. The input device 210 may include a keyboard, a mouse, avoice-controlled input device, a touch sensitive input device (e.g., atouch screen), and/or the like for receiving user inputs (e.g., from amedical professional) to the control unit 200.

The processor 202 may be configured to perform various tasks associatedwith automating a medical procedure for a patient. These tasks mayinclude, for example, determining the characteristics and/or identity ofa patient based on images generated by an image capturing device (e.g.,the image capturing device 104 in FIG. 1), retrieving medicalinformation of the patient from one or more medical record repositories(e.g., the repository 116 in FIG. 1), providing positioning assistanceto the patient based on physical attributes of the patient reflected inthe received images, making autonomous diagnosis and/or treatmentplanning for the patient based on past and present information gatheredabout the patient, interacting with the patient or medical professionalsduring the medical procedure to ensure all parties involved are properlyinformed, etc.

One or more of the aforementioned tasks may be accomplished utilizingartificial intelligence techniques such as machine learned decisionmodels and AI-based imaging processing techniques. In a first aspect,the processor 202 may be configured to process received images through apreprocessing stage during which the processor may discard images thatare of poor quality and/or convert qualified images into a suitableformat for further processing. The processor 202 may also prepare theimages in ways that would reduce the complexity of further processing.Such preparation may include, for example, converting color images tograyscale, resize the images into unified dimensions, and/or the like.

In a second aspect, the processor 202 may include or may be coupled to afeature database 212 configured to store visual representations of knownfeatures of a patient (e.g., facial features, body shapes, positions,poses, walking patterns, etc.) and/or known patterns (e.g., X-ray or CTscan patterns) that indicate certain medical conditions. The knownfeatures or patterns may be pre-computed and stored in the featuredatabase 212 based on imagery data collected from various sourcesincluding, for example, pictures taken during the patient's past visitsto a medical facility and/or historical medical records stored in arepository (e.g., the repository 116 shown in FIG. 1). The featuredatabase 212 may be communicatively coupled to the processor 202 andused by the processor for identifying a patient or making a diagnosis.For example, in response to receiving images of a patient from the imagecapturing device 104, the processor 202 may analyze the images (e.g., ata pixel level) to extract a set features present in the images. Thefeatures may relate to a variety of attributes of the patient includingbut not limited to body contours, facial features, walking patterns,poses, etc. Each feature may correspond a combination of structures(e.g., points, edges, objects, etc.) arranged in a specific manner inthe images, and as such may be identified based on the presence of oneor more keypoints. These keypoints may include but are not limited to,for example, points at which the direction of the boundary of an objectchanges abruptly, intersection points between two or more edge segments,etc. The keypoints may be characterized by well-defined positions in theimage space and/or stability to illumination/brightness perturbations.Accordingly, the keypoints may be identified based on image derivatives,edge detection, curvature analysis, and/or the like. And onceidentified, the keypoints and/or the feature represented by thekeypoints may be described with a feature descriptor or feature vector.In an example implementation of such feature descriptors or vectors,information related to the feature (e.g., appearance of the localneighborhood of each keypoint) may be represented by (e.g., encodedinto) a series of numerical values stored in the feature descriptor orvector. The descriptor or vector may then be used as a “fingerprint” fordifferentiating one feature from another or matching one feature withanother.

In a third aspect, the processor 202 may include, be coupled to, orotherwise utilize a machine learning model (e.g., in addition to orinstead of the feature database) configured to perform one or more ofthe tasks described herein. In an example implementation, the machinelearning model may be based on or acquired through a neural network 212(e.g., in addition to or instead of the feature database). The neuralnetwork 212 may include a convolutional neural network (CNN) and/or adeep neural network (DNN) that comprises multiple layers (e.g., an inputlayer, one or more convolutional layers, one or more non-linearactivation layers, one or more pooling layers, one or more fullyconnected layers, and/or an output layer). Each of the layers maycorrespond to a plurality of filters (or kernels), and each filter maybe designed to detect a specific type of visual features or patterns.The filters may be associated with respective weights that, when appliedto an input, produce an output indicating whether certain visualfeatures or patterns have been detected. The weights associated with thefilters may be learned by the neural network 212 through a trainingprocess that comprises inputting a large number of images from atraining dataset to the neural network (e.g., in a forward pass),calculating losses resulting from the weights currently assigned to thefilters (e.g., based on a loss function such as a margin based lossfunction), and updating (e.g., in a backward pass) the weights assignedto the filters so as to minimize the losses (e.g., based on stochasticgradient descent). Once trained, the neural network 212 may be able totake an image at the input layer, extract and/or classify visualfeatures or patterns from the image, and provide an indication at theoutput layer for whether the input image matches that of a known personor a known scan pattern associated with a medical condition.

FIG. 3 is a simplified diagram illustrating example functional modulesthat may be comprised in a processor 300 (e.g., the processor 202) inaccordance with examples provided herein. It should be noted that, whenreferenced herein, the term “module” does not mean or imply thatfunctionalities described in association with the module are implementedseparately from or independently of other functionalities of theprocessor 300. Rather, the term is merely used for the convenience ofdescription and not meant to indicate any structural limitations ordesign preferences.

As shown in FIG. 3, the processor 300 may include a control logic module302, a patient identification module 304, a positioning assistancemodule 306, a diagnosis module 308, and/or a treatment planning module310. The control logic module 302 may be responsible for controlling thegeneral operation of the processor 300 such as receiving or respondingto user inputs, while the other modules may be configured to performspecific functions relating to the automation of healthcare services asdescribed herein. For example, the patient identification module 304 maybe configured to receive images generated by an image capturing device(e.g., the image capturing device 104) and extract features from theimages to determine the identity and/or characteristics (e.g., physicalcharacteristics) of the patient, e.g., using one or more of the AI-basedon image recognition techniques described herein.

The positioning assistance module 304 may be responsible for monitoringthe position of the patient during a medical procedure and providingguidance to the patient so as to help the patient get into a desiredposition or pose. Such desired position or pose may be determined, forexample, based on a protocol associated with the medical procedureand/or physical characteristics of the patient determined from theimages acquired by the image capturing device 104. For example, thepositioning assistance module 304 may determine, based on a patientidentity provided by the patient identification module 304, that a scanprocedure is to be performed for the patient and that the scan locationis in the chest area of the patient. The positioning assistance module304 may further determine the height of the patient based on one or moreimages acquired by the image capturing device 104. Combining thesepieces of information, the positioning assistance module 304 maydetermine and instruct the patient about a proper position or pose totake so that the scan can be accurately performed in the chest area ofthe patient. In the process, the positioning assistance module 304 mayalso generate control signals for adjusting one or more operatingparameters of the medical scanner (e.g., the height of a scan bed, theorientation of a scanner, etc.) to help the patient get into the desiredposition. The control signals may be digital and/or analog controlsignals and may be transmitted to the medical scanner via wired orwireless means.

The positioning assistance module 304 may also be responsible forevaluating the readiness of a patient during a medical procedure. Forinstance, after the patient has been instructed about a desirableposition to take, the positioning assistance module 304 may furtherdetermine, based on multiple images of the patient taken after theinstructions have been given, that the patient has entered into andremained steady in the desired position. The positioning assistancemodule 304 may then initiate the medical procedure or inform a medicalprofessional that the patient is ready for the procedure.

The diagnosis module 308 may be responsible for making automaticdiagnoses for a patient (e.g., as part of initial screening) based oninformation collected during a medical procedure (e.g., based on one ormore scan images of the patient). The automatic diagnosis may be made,for example, utilizing one or more of the AI-based image recognitiontechniques described herein to identify abnormalities in the scan imagesand determine the medical conditions that may be associated with theabnormalities. Upon obtaining the diagnostic results, the diagnosismodule 308 may prioritize or indicate the results as critical, urgent,non-urgent, normal or uncertain, and report the results to a medicalprofessional for analysis or review.

The treatment planning module 310 may be responsible for devisingtreatment plans for a patient based on machine-learned treatmentstrategies and information collected about the patient. The treatmentstrategies (or models) may be learned (e.g., using a neural network)from databases of clinically accepted plans, for example, by usinggeometric and dosimetric features contained in the plans to predict arange of achievable dose deposition for new patients. The informationcollected about the patient may include physical characteristics (e.g.,body shape, weight, etc.) of the patient, medical history of thepatient, and/or diagnoses of the patient. As described herein, thephysical characteristics of the patient may be determined based onimages acquired by the image capturing device 104, the medical historyof the patient may be retrieved from a medical record repository such asthe repository 116, and the diagnoses of the patient may be obtainedfrom the diagnosis module 308. Once collected, one or more pieces of theinformation may be provided as inputs to the machine-learned treatmentmodels (e.g., to a neural network) to derive a suitable plan for thepatient at the output of the model (e.g., the neural network).

FIG. 4 is a flow diagram illustrating a method 400 that may beimplemented by an automated healthcare system described herein (e.g.,the system 100 of FIG. 1). For simplicity of explanation, the operationsin the method 400 are depicted and described herein with a specificorder. It should be appreciated, however, that these operations mayoccur in various orders and/or concurrently, and with other operationsnot presented and described herein. Furthermore, not all illustratedoperations may be required to implement the method disclosed herein.

The method 400 may be started by a control unit of the automatedhealthcare system (e.g., the control unit 110 or 200 shown in FIG. 1 orFIG. 2) at 402. At 404, the control unit may receive one or more imagesof a patient from an image capturing device (e.g., the image capturingdevice 104 of FIG. 1) located in or around a medical facility such as ahospital, a physician's office, a scan or treatment room, etc. Theimages may be in a variety of formats including camera photos, thermalimages, radar images, and/or the other types of imagery that contain arepresentation of the patient. At 406, the control unit may analyze thereceived images and extract features (or patterns) from the images thatcollectively indicate the identity and/or characteristics (e.g., bodyshape, height, etc.) of the patient. The analysis of the images may beconducted at a pixel level (e.g., pixel by pixel, by groups of pixels,etc.) and/or utilizing a neural network or feature database. Onceextracted, the features may be compared with known features of patientsto determine whether a match can be found. If a match is found, thecontrol unit may, at 408, further verify the identity of the patientbased on other information the control unit can gather about thepatient. For instance, using the identity determined from the images,the control unit may retrieve additional information regarding thepatient from a record repository. The additional information mayinclude, for example, height, weight, body shape, age, and/or gender ofthe patient. The control unit may compare the additional informationwith the physical characteristics of the patient determined from theimages and determine whether there is any error in the identification ofthe patient.

At 410, the control unit may start preparing the patient for an upcomingmedical procedure. For example, the control unit may determine, based onthe characteristics (e.g., physical characteristics) of the patientidentified from the images and/or a protocol designed for the medicalprocedure, a desired patient position for the medical procedure and/oran operating parameter of the medical equipment involved in theprocedure. Subsequently, the control unit may instruct the patient(e.g., by sending visual and/or audio instructions to the patient) aboutthe desired position and/or ways to maneuver into the desired position.The control unit may also generate and transmit control signals to themedical equipment (e.g., to a controller of the medical equipment) toeffectuate the operating parameter (e.g., the height of a scan bed)needed for the medical procedure. The control signals may be digitaland/or analog control signals and may be transmitted to the medicalequipment via wired or wireless means.

At 412, the control unit may monitor the status and/or movements of thepatient before and during the medical procedure. For example, thecontrol unit may determine the readiness of the patient for the medicalprocedure by analyzing multiple images of the patient gathered over atime period during the preparation process. The control unit may extractpositional information of the patient from each of the images andcompare the positional information across multiple images to ensure thatthe patient has remained steady in a desired position for the medicalprocedure. Similarly, the control unit may identify movements of thepatient by analyzing multiple images of the patient collected during themedical procedure to ensure that the patient has followed instructions(e.g., positioning instructions) provided by the control unit or amedical professional overseeing the medical procedure. The control unitmay provide feedback regarding the patient's status and/or movements tothe medical professional. The control unit may also provide instructionsto the patient to assist the patient before and during the medicalprocedure.

At 414, the control unit may provide automated diagnosis and/ortreatment planning for the patient, for example, utilizing AI-basedprediction models and/or methods described herein.

While this disclosure has been described in terms of certain embodimentsand generally associated methods, alterations and permutations of theembodiments and methods will be apparent to those skilled in the art.Accordingly, the above description of example embodiments does notconstrain this disclosure. Other changes, substitutions, and alterationsare also possible without departing from the spirit and scope of thisdisclosure. In addition, unless specifically stated otherwise,discussions utilizing terms such as “segmenting”, “analyzing”,“determining”, “enabling”, “identifying,” “modifying” or the like, referto the actions and processes of a computer system, or similar electroniccomputing device, that manipulates and transforms data represented asphysical (e.g., electronic) quantities within the computer system'sregisters and memories into other data represented as physicalquantities within the computer system memories or other such informationstorage, transmission or display devices.

It is to be understood that the above description is intended to beillustrative, and not restrictive. Many other implementations will beapparent to those of skill in the art upon reading and understanding theabove description. The scope of the disclosure should, therefore, bedetermined with reference to the appended claims, along with the fullscope of equivalents to which such claims are entitled.

What is claimed is:
 1. A system for providing automated healthcare services, comprising: at least one image capturing device configured to capture one or more images of a patient; and a control unit configured to: receive the one or more images of the patient generated by the at least one image capturing device; analyze the one or more images to identify at least one characteristic of the patient; complete, automatically, one or more aspects of a medical procedure for the patient in accordance with the at least one characteristic of the patient, wherein the one or more aspects of the medical procedure that are automatically completed include remotely controlling a medical device associated with the medical procedure or remotely providing instructions regarding the medical procedure to the patient; and provide feedback regarding the patient or the medical procedure to a receiving device isolated from the patient.
 2. The system of claim 1, wherein the at least one image capturing device comprises a digital camera or a thermal sensor, and the one or more images of the patient comprise a photo of the patient taken by the digital camera or a thermal image of the patient generated by the thermal sensor.
 3. The system of claim 1, wherein remotely controlling the medical device comprises transmitting a control signal to the medical device for adjusting an operating parameter of the medical device.
 4. The system of claim 3, wherein the operating parameter of the medical scanner relates to at least one of a scan location, a scan direction or a scan range.
 5. The system of claim 1, wherein the instructions remotely provided to the patient comprise instructions for positioning the patient for the medical procedure.
 6. The system of claim 1, wherein the control unit is further configured to determine a spatial relationship between the at least one image capturing device and the medical device associated with the medical procedure, the control unit further configured to complete the one or more aspects of the medical procedure based on the spatial relationship.
 7. The system of claim 6, wherein the at least one image capturing device is associated with a first coordinate system, the medical device is associated with a second coordinate system, and the control unit being configured to determine the spatial relationship between the at least one image capturing device and the medical device associated with the medical procedure comprises the control unit being configured to convert coordinates in the first coordinate system to coordinates in the second coordinate system.
 8. The system of claim 6, wherein the control unit is further configured to determine a parameter associated with the medical procedure based on the spatial relationship between the at least one image capturing device and the medical device, the control unit further configured to overlay the one or more images of the patient with an indication of the parameter and cause a representation of the overlaid one or more images to be displayed on the receiving device.
 9. The system of claim 1, wherein the one or more aspects of the medical procedure are automatically completed for the patient without requiring a medical professional to make physical contract with the patient.
 10. The system of claim 1, wherein the receiving device is located in a separate room from the patient.
 11. The system of claim 1, wherein the control unit is configured to: extract, from the one or more images of the patient, positional information of the patient relating to the medical procedure; and determine whether the patient is in a ready position for the medical procedure based on the positional information extracted from the one or more images.
 12. The system of claim 11, wherein the control unit is configured to acquire the one or more images of the patient over a period of time, determine a position of the patient in each of the one or more images, and determine whether the patient is in the ready position by comparing the position of the patient in each of the one or more images.
 13. The system of claim 12, wherein the instructions remotely provided to the patient include position adjustment instructions to the patient in response to determining that the patient is not in the ready position for the medical procedure.
 14. The system of claim 13, wherein the control unit is further configured to determine, based on the one or more images captured by the at least one image capturing device, whether the patient has followed the adjustment instructions.
 15. The system of claim 1; wherein the control unit is further configured to determine, based on the one or more images of the patient, an identity of the patient.
 16. The system of claim 15, wherein the control unit is further configured to retrieve medical information of the patient based on the identity of the patient.
 17. The system of claim 1, wherein the control unit is configured to use a neural network to identify the at least one characteristic of the patient from the one or more images of the patient.
 18. The system of claim 1, wherein the at least one characteristic of the patient includes at least one of a height of the patient or a body shape of the patient.
 19. A device for providing automated healthcare services, comprising: at least one processor configured to: receive one or more images of a patient generated by an image capturing device; analyze the one or more images to identify at least one characteristic of the patient; complete, automatically, one or more aspects of a medical procedure for the patient in accordance with the at least one characteristic of the patient, wherein the one or more aspects of the medical procedure that are automatically completed include remotely controlling a medical device associated with the medical procedure or remotely providing instructions regarding the medical procedure to the patient; and provide feedback regarding the patient or the medical procedure to a receiving device isolated from the patient.
 20. A method for providing automated healthcare services, the method comprising: receiving one or more images of a patient generated by an image capturing device; analyzing the one or more images to identify at least one characteristic of the patient; completing, automatically, one or more aspects of a medical procedure for the patient in accordance with the at least one characteristic of the patient, wherein the one or more aspects of the medical procedure that are automatically completed for the patient include remotely controlling a medical device associated with the medical procedure or remotely providing instructions regarding the medical procedure to the patient; and providing feedback regarding the patient or the medical procedure to a receiving device isolated from the patient. 